Skip to content

tests: Add example file validation test suite.#213

Merged
nedseb merged 2 commits intomainfrom
feat/validate-examples
Mar 23, 2026
Merged

tests: Add example file validation test suite.#213
nedseb merged 2 commits intomainfrom
feat/validate-examples

Conversation

@nedseb
Copy link
Copy Markdown
Contributor

@nedseb nedseb commented Mar 23, 2026

Closes #212

Summary

Add tests/test_examples.py — a test suite that validates all example files in lib/*/examples/. Can be run in CI alongside existing mock tests.

What it checks

For each example file (51 files across 15 drivers):

Check Description
Syntax File must compile as valid Python
Final newline File must end with \n
No bare except Must use except Exception: per project convention
Method calls Calls on driver objects must match methods in device.py

Results on current main

  • 189 passed, 2 failed (pre-existing bugs), 17 skipped
  • Skips are for examples that don't import a driver (e.g., ssd1327 examples use framebuf directly)

Pre-existing bugs detected

File Bug
lib/lis2mdl/examples/magnet_test.py Calls mag.read_reg() — method is _read_reg() (private)
lib/wsen-pads/examples/test.py Calls sensor._read_u8() — method doesn't exist

These are tracked in issue #212 and should be fixed separately.

CI integration

Add to the test workflow:

- run: python -m pytest tests/test_examples.py -v

@chatgpt-codex-connector
Copy link
Copy Markdown

chatgpt-codex-connector Bot commented Mar 23, 2026

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@nedseb nedseb requested a review from Copilot March 23, 2026 06:57
This was referenced Mar 23, 2026
Closed
Closed
Copilot AI reviewed Mar 23, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new pytest suite to validate Python example files under lib/*/examples/, aiming to catch syntax issues, missing final newlines, bare except:, and calls to driver methods that don’t exist in the corresponding device.py.

Changes:

  • Introduces tests/test_examples.py to discover and validate all driver example scripts.
  • Implements AST-based checks for bare except: and driver method-call validity.
  • Adds file-level checks for Python compilation and final newline presence.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_examples.py Outdated
Comment thread tests/test_examples.py Outdated
Comment thread tests/test_examples.py Outdated
Comment thread tests/test_examples.py Outdated
Comment thread tests/test_examples.py Outdated
@nedseb
Copy link
Copy Markdown
Contributor Author

nedseb commented Mar 23, 2026

All 5 Copilot comments addressed in fde1f76:

  1. Unused imports — Removed re, importlib, inspect and MICROPYTHON_MODULES.
  2. SyntaxError in device.py — Now calls pytest.fail() with a clear message instead of silently returning empty set.
  3. SyntaxError in test_method_calls — Added try/except SyntaxError: return guard (defers to test_syntax_valid).
  4. Overly broad allowlist — Replaced with a minimal non_driver_methods set. Additionally, method discovery now scans all .py files in the module (not just device.py) to cover pin.py etc.
  5. Non-deterministic module dir selection — Now prefers the directory matching the driver name, falls back to sorted order.

@nedseb nedseb added the enhancement New feature or request label Mar 23, 2026
@nedseb nedseb merged commit 9758c2b into main Mar 23, 2026
3 checks passed
@nedseb nedseb mentioned this pull request Mar 23, 2026
Merged
@nedseb nedseb deleted the feat/validate-examples branch March 25, 2026 10:39
@semantic-release-updater
Copy link
Copy Markdown

semantic-release-updater Bot commented Mar 25, 2026

🎉 This PR is included in version 0.0.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

@nedseb nedseb

Labels

enhancement New feature or request released

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

ci: Add example file validation to CI pipeline.

2 participants

@nedseb